Here are some thoughts from a couple of years ago when genAI first hit us, plus some thoughts about where we are going with it.
Background Narrative:
Can evaluators find a niche in auditing whether AI applications are trustworthy, culture-aware, valid and transparent?
We can thank Judea Pearl for promoting the insight that if you want to thrive in this world, you have to understand causality natively. We humans make causal connections from an early age. We wouldn't survive long if we didn't.
Causal mapping – the process of identifying and synthesising causal claims within documents – is about to become much more accessible to evaluators. At Causal Map Ltd, we use causal mapping to solve evaluation problems, for example to create “empirical theories of change” or to trace evidence of the impact of inputs on outcomes.
🏭 When machines replaced much manual labour, white-collar workers thought "I'm ok, my job is much harder to mechanise".
What would Wittgenstein say about this? We can use his concepts of language games and also family resemblances.
Matthew Clifford says: “There are no AI-shaped holes lying around”. That is how he reconciles "the facts that (a) AI is already powerful and (b) it’s having relatively little impact so far Making AI work today requires ripping up workflows and rebuilding for AI. This is hard and painful to do…"